Thermal imaging camera with range detection

ABSTRACT

A thermal imaging camera may be used to capture a visible-light (VL) image and an infrared (IR) image. In some examples, the camera includes a range imaging camera module that captures the VL and an infrared camera module that captures the IR image. In such examples, the VL image may include a plurality of different portions that each correspond to a different portion of the scene and distance-to-target data associated with each of the different portions of the scene. The camera may align each of the plurality of different portions of the VL image based on the distance-to-target data associated with corresponding portions of the scene so as to correct a parallax error between the VL image and the IR image. The camera may then concurrently display the VL image in alignment with the IR image.

TECHNICAL FIELD

This disclosure relates to thermal imaging cameras and, moreparticularly, to thermal imaging cameras with range detectioncapabilities.

BACKGROUND

Thermal imaging cameras are used in a variety of situations. Forexample, thermal imaging cameras are often used during maintenanceinspections to thermally inspect equipment. Example equipment mayinclude rotating machinery, electrical panels, or rows of circuitbreakers, among other types of equipment. Thermal inspections can detectequipment hot spots such as overheating machinery or electricalcomponents, helping to ensure timely repair or replacement of theoverheating equipment before a more significant problem develops.

Thermal imaging cameras typically generate thermal images by capturinginfrared energy emitted by an object and then translating the capturedinfrared energy into an image representative of a temperature profileacross the object. Depending on the configuration of the camera, thethermal imaging camera may also generate a visible light image of thesame object. The camera may display the infrared image and the visiblelight image in a coordinated manner, for example, to help an operatorfocus and interpret the thermal image generated by the thermal imagingcamera. Unlike visible light images which generally provide goodcontrast between different objects, it is often difficult to recognizeand distinguish different features in a thermal image as compared to thereal-world scene. For this reason, an operator may rely on a visiblelight image to help interpret and focus the thermal image.

In applications where a thermal imaging camera is configured to generateboth a thermal image and a visual light image, the camera may includetwo separate sets of optics for focusing the thermal image and visuallight image. While the separate sets of optics can provide independentfocusing control, the different positional arrangement of each set ofoptics can create a parallax, or shift, between the two images. Theparallax may be proportional to the distance between each set of optics.The parallax may also be proportional to the distance between thethermal imaging camera and the object being observed. Accordingly, beingable to accurately determine the distance between the thermal imagingcamera and the object being observed during operation may be useful toresolve the parallax between the two images. Knowledge of the distancebetween a thermal imaging camera and an object being observed can beused for other purposes as well.

SUMMARY

In general, this disclosure is directed to apparatuses and techniquesfor capturing both a visible light image of a scene and an infraredimage of the same scene. In some examples, a thermal imaging cameraincludes a range imaging camera module that is configured to capture avisible light image that includes distance-to-target data associatedwith each of a plurality of different portions of a target scene. Thethermal imaging camera may align each of a plurality of differentportions of the visible light image based on the distance-to-target dataassociated with corresponding portions of the scene so as to correct aparallax error between the visible light image and the infrared image.In some examples, the thermal imaging camera can also concurrentlydisplay the visible light image in alignment with the infrared image. Byaligning each of a plurality of different portions of a visible lightimage based on distance-to-target data associated with differentportions of a scene, the accuracy with which the visible light image andinfrared image can be concurrently displayed may be improved as comparedto if all the portions of the visible light image were aligned together,e.g., by shift all portions of the image a fixed amount.

In one example, this disclosure describes a camera that includes a rangeimaging camera module, an infrared camera module, a display, and aprocessor. The range imaging camera module is configured to capture avisible-light (VL) image of a scene along a first optical axis, wherethe VL image includes a plurality of different portions that eachcorrespond to a different portion of the scene and distance-to-targetdata associated with each of the different portions of the scene. Theinfrared camera module is configured to capture an infrared (IR) imageof the scene along a second optical axis, the second optical axis beingoffset from the first optical axis so that the IR image of the scene isfrom a different point of view than the VL image thereby causing aparallax error. According to the example, the processor is configured toalign each of the plurality of different portions of the VL image basedon the distance-to-target data associated with corresponding portions ofthe scene so as to correct the parallax error between the VL image andthe IR image, and control the display to concurrently display at least aportion of the VL image in alignment with at least a portion of the IRimage.

In another example, a method is described that includes receivingvisible-light (VL) image data representative of a VL image of a scenecaptured via a range imaging camera module along a first optical axis,the VL image including a plurality of different portions that eachcorrespond to a different portion of the scene and distance-to-targetdata associated with each of the different portions of the scene. Themethod also includes receiving infrared (IR) image data representativeof an IR image of the scene captured via an IR camera module along asecond optical axis, the second optical axis being offset from the firstoptical axis so that the IR image of the scene is from a different pointof view than the VL image thereby causing a parallax error. According tothe example, the method further includes aligning each of the pluralityof different portions of the VL image based on the distance-to-targetdata associated with corresponding portions of the scene so as tocorrect the parallax error between the VL image and the IR image, andconcurrently displaying at least a portion of the VL image in alignmentwith at least a portion of the IR image.

In another example, a computer-readable storage medium is described thatincludes instructions that cause a programmable processor to receivevisible-light (VL) image data representative of a VL image of a scenecaptured via a range imaging camera module along a first optical axis,the VL image including a plurality of different portions that eachcorrespond to a different portion of the scene and distance-to-targetdata associated with each of the different portions of the scene. Thecomputer-readable storage medium also includes instructions that causethe programmable processor to receive infrared (IR) image datarepresentative of an IR image of the scene captured via an IR cameramodule along a second optical axis, the second optical axis being offsetfrom the first optical axis so that the IR image of the scene is from adifferent point of view than the VL image thereby causing a parallaxerror. According to the example, the computer-readable storage mediumincludes instructions that cause the programmable processor to aligneach of the plurality of different portions of the VL image based on thedistance-to-target data associated with corresponding portions of thescene so as to correct the parallax error between the VL image and theIR image, and control a display to concurrently display at least aportion of the VL image in alignment with at least a portion of the IRimage.

The details of one or more examples are set forth in the accompanyingdrawings and the description below. Other features, objects, andadvantages will be apparent from the description and drawings, and fromthe claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a perspective front view of an example thermal imaging camera.

FIG. 2 is a perspective back view of the thermal imaging camera of FIG.1.

FIG. 3 is a functional block diagram illustrating example components ofthe thermal imaging camera of FIGS. 1 and 2.

FIGS. 4A and 4B are schematic illustrations of alignment of differentportions of a visible light image and an infrared image.

FIG. 5 is a flow chart of an example method of concurrently displaying avisible light image and an infrared image.

FIG. 6 is a conceptual illustration of an example picture-in-picturetype concurrent display of a visual image and an infrared image.

DETAILED DESCRIPTION

The following detailed description is exemplary in nature and is notintended to limit the scope, applicability, or configuration of theinvention in any way. Rather, the following description provides somepractical illustrations for implementing examples of the presentinvention. Examples of constructions, materials, dimensions, andmanufacturing processes are provided for selected elements, and allother elements employ that which is known to those of ordinary skill inthe field of the invention. Those skilled in the art will recognize thatmany of the noted examples have a variety of suitable alternatives.

A thermal imaging camera may be used to detect heat patterns across ascene under observation. The thermal imaging camera may detect infraredradiation given off by the scene and convert the infrared radiation intoan infrared image indicative of the heat patterns. In some examples, thethermal imaging camera may also capture visible light from the scene andconvert the visible light into a visible light image. Depending on theconfiguration of the thermal imaging camera, the camera may includeinfrared optics to focus the infrared radiation on an infrared detectorand visible light optics to focus the visible light on a visible lightdetector. The infrared optics and visible light optics may be offsetfrom one another so as to create a parallax, or shift, between theinfrared image and the visible light image generated by the camera. Ifthe thermal imaging camera displays the infrared image and visible lightimage simultaneously, e.g., in overlapping alignment, the two images maynot properly align with one another unless the camera corrects for theparallax between the two images.

In accordance with the techniques described in some examples of thisdisclosure, a thermal imaging camera is provided that is configured toresolve a parallax shift between a visible light image and an infraredimage and concurrently display the visible light image in registrationwith the infrared image. In some examples, the thermal imaging cameraincludes a range imaging camera module that is configured to capture avisible light image that includes distance-to-target data associatedwith each of a plurality of different portions of a target scene. Forexample, the thermal imaging camera may include a plurality of visiblelight sensor elements, and the range imaging camera module may beconfigured to determine a distance-to-target value associated with eachof the plurality of visible light sensor elements. Thedistance-to-target value may represent a distance between the rangeimaging camera module and a point in a target scene captured by aspecific visible light sensor element. The thermal imaging camera mayalign each of a plurality of different portions of the visible lightimage based on the distance-to-target data associated with correspondingportions of the scene so as to correct the parallax error.

In contrast to a thermal imaging camera that resolves a parallax betweena visible light image and an infrared image based on a singledistance-to-target value (e.g., a distance between an infrared cameralens and a single feature in a scene under observation), a thermalimaging camera according to some examples of the disclosure may resolvea parallax between a visible light image and an infrared image based ona plurality of different distance-to-target values associated withdifferent portions of a scene under observation. Depending on theconfiguration of the thermal imaging camera, a feature in the scene thatis relatively close to the thermal imaging camera may exhibit moreparallax, or shift, between the visible light image and infrared imagethan a feature in the scene that is farther away from the thermalimaging camera. Accordingly, when resolving the parallax in theseexamples, the thermal imaging camera may shift the portion of thevisible light image or infrared image capturing the relative closefeature in the scene more than the portion of the visible light image orinfrared image capturing the farther away feature. This may improve theaccuracy with which the visible light image and infrared image can beconcurrently displayed as compared to if all the features captured bythe visible light image or infrared image were shifted the same amount.

An example method of displaying an infrared image with a visible lightimage, and an example concurrent display of an infrared image with avisible light image will be described in greater detail with respect toFIGS. 5 and 6. However, an example thermal imaging camera will first bedescribed with respect to FIGS. 1-3.

FIGS. 1 and 2 show front and back perspective views, respectively of anexample thermal imaging camera 10, which includes a housing 12, aninfrared lens assembly 14, a visible light lens assembly 16, a display18, and a trigger control 20. Housing 12 houses the various componentsof thermal imaging camera 10. Infrared lens assembly 14 receivesinfrared radiation from a scene and focuses the radiation on an infrareddetector for generating an infrared image of a scene. Visible light lensassembly 16 receives visible light from a scene and focuses the visiblelight on a visible light detector for generating a visible light imageof the same scene. Thermal imaging camera 10 captures the visible lightimage and/or the infrared image in response to depressing triggercontrol 20. In addition, thermal imaging camera 10 controls display 18to display the infrared image and the visible light image generated bythe camera, e.g., to help an operator thermally inspect a scene.

As described in greater detail below, thermal imaging camera 10 includesa infrared camera module that is configured to capture an infrared imageof a scene and a range imaging camera module that is configured tocapture a visible light image of the same scene. The infrared cameramodule may receive infrared radiation projected through infrared lensassembly 14 and generate therefrom infrared image data. In addition, therange imaging camera module may receive light projected through visiblelight lens assembly 16 and generate therefrom visible light data. Insome examples, the range imaging camera module is also configured togenerate distance-to-target data for a plurality of different portionsof a scene as part of the visible light data. For example, the rangeimaging camera module may generate distance-to-target data for each of aplurality of different visible light sensor elements, where each sensorcaptures a different portion of a scene under observation. Thermalimaging camera 10 may correct a parallax between the visible light imageand the infrared image based on the distance-to-target data or use thedistance-to-target data to perform other processing functions.

In operation, thermal imaging camera 10 detects heat patterns in a sceneby receiving energy emitted in the infrared-wavelength spectrum from thescene and processing the infrared energy to generate a thermal image.Thermal imaging camera 10 may also generate a visible light image of thesame scene by receiving energy in the visible light-wavelength spectrumand processing the visible light energy to generate a visible lightimage. In some examples, thermal imaging camera 10 collects or capturesthe infrared energy and visible light energy substantiallysimultaneously (e.g., at the same time) so that the visible light imageand the infrared image generated by the camera are of the same scene atsubstantially the same time. In these examples, the infrared imagegenerated by thermal imaging camera 10 is indicative of localizedtemperatures within the scene at a particular period of time while thevisible light image generated by the camera is indicative of the samescene at the same period of time. In other examples, thermal imagingcamera may capture infrared energy and visible light energy from a sceneat different periods of time.

Thermal imaging camera 10 captures infrared energy through infrared lensassembly 14 and visible light energy through visible light lens assembly16. Infrared lens assembly 14 and visible light lens assembly 16 canhave a number of different orientations relative to housing 12. In someexamples, infrared lens assembly 14 and visible light lens assembly 16are offset from one another, e.g., in a fixed spatial relationshiprelative to housing 12, so as to create a parallax error between theinfrared image and the visible light image generated by thermal imagingcamera 10. For instance, in one example, infrared lens assembly 14 andvisible light lens assembly 16 are horizontally offset from one another,e.g., in a coplanar relationship. In another example, infrared lensassembly 14 and visible light lens assembly 16 are vertically offsetfrom one another. In the example of FIG. 1, infrared lens assembly 14and visible light lens assembly 16 are vertically offset from oneanother in generally parallel arrangement. Infrared lens assembly 14 andvisible light lens assembly 16 are generally parallel to one another inthe example of FIG. 1 in that both lens assemblies are generally pointedtowards the same scene under observation.

Infrared lens assembly 14 includes at least one lens that focusesinfrared energy on an infrared detector for generating a thermal image.Infrared lens assembly 14 defines an infrared optical axis 22 whichpasses through the center of curvature of the at least one lens of theassembly. During operation, infrared energy is directed through thefront of the lens and focused on an opposite side of the lens. Infraredlens assembly 14 can include a single lens or a plurality of lenses(e.g., two, three, or more lenses), which may arranged in series.

In some examples, infrared lens assembly 14 also includes a focusadjustment mechanism for changing the focus of the infrared optics. Theinfrared focus adjustment mechanism may be a manual focus adjustmentmechanism, or the focus adjustment mechanisms may automatically adjustthe focus of the infrared optics. In the example of FIGS. 1 and 2,thermal imaging camera 10 includes a rotatable focus ring 24 formanually adjusting a focus of the infrared optics. Rotation of focusring 24 may move an infrared lens so as to manually adjust the focus ofthe infrared optics. In other examples, thermal imaging camera 10 mayautomatically adjust the focus of the at least one infrared lens ofinfrared lens assembly 14. For example, as described in greater detailbelow, thermal imaging camera 10 may automatically adjust a focus the atleast one infrared lens of infrared lens assembly 14 based ondistance-to-target data generated by a range imaging camera module.

Visible light lens assembly 16 also includes at least one lens thatfocuses visible light energy on a visible light detector for generatinga visible light image. Visible light lens assembly 16 defines a visiblelight optical axis 26 which passes through the center of curvature ofthe at least one lens of the assembly. Visible light energy projectsthrough a front of the lens and focuses on an opposite side of the lens.As with the infrared lens assembly 14, visible light lens assembly 16can include a single lens or a plurality of lenses (e.g., two, three, ormore lenses) arranged in series. In addition, visible light lensassembly 16 can include a focus adjustment mechanism for changing thefocus of the visible light optics. In examples in which visible lightlens assembly 16 includes a focus adjustment mechanism, the focusadjustment mechanism may be a manual adjustment mechanism or anautomatic adjustment mechanism.

Thermal imaging camera 10 can be configured to display a thermal imageof a scene and/or a visible light image of the same scene. For thisreason, thermal imaging camera 10 may include a display. In the exampleof FIGS. 1 and 2, thermal imaging camera 10 includes display 18, whichis located on the back of housing 12 opposite infrared lens assembly 14and visible light lens assembly 16. Display 18 may be configured todisplay a visible light image, an infrared image, and/or a blended imagethat is a simultaneously display of the visible light image and theinfrared image. In different examples, display 18 may be remote (e.g.,separate) from infrared lens assembly 14 and visible light lens assembly16 of thermal imaging camera 10, or display 18 may be in a differentspatial arrangement relative to infrared lens assembly 14 and/or visiblelight lens assembly 16. Therefore, although display 18 is shown behindinfrared lens assembly 14 and visible light lens assembly 16 in FIG. 2,other locations for display 18 are possible.

Thermal imaging camera 10 can include a variety of user input media forcontrolling the operation of the camera and adjusting different settingsof the camera. Example control functions may include adjusting the focusof the infrared and/or visible light optics, opening/closing a shutter,capturing an infrared and/or visible light image, or the like. In theexample of FIGS. 1 and 2, thermal imaging camera 10 includes adepressible trigger control 20 for capturing an infrared and visiblelight image, and buttons 28 for controlling other aspects of theoperation of the camera. A different number or arrangement of user inputmedia are possible, and it should be appreciated that the disclosure isnot limited in this respect. For example, thermal imaging camera 10 mayinclude a touch screen display 18 which receives user input bydepressing different portions of the screen.

FIG. 3 is a functional block diagram illustrating components of anexample of thermal imaging camera 10, which includes an infrared cameramodule 100, a range imaging camera module 102, a display 104, aprocessor 106, a user interface 108, a memory 110, and a power supply112. Processor is communicatively coupled to infrared camera module 100,range imaging camera module 102, display 104, user interface 108, andmemory 110. Power supply 112 delivers operating power to the variouscomponents of thermal imaging camera 10 and, in some examples, mayinclude a rechargeable or non-rechargeable battery and a powergeneration circuit.

During operation of thermal imaging camera 10, processor 106 controlsinfrared camera module 100 and range imaging camera module 102 with theaid of instructions associated with program information that is storedin memory 110 to generate a visible light image and an infrared image ofa target scene. Processor 106 further controls display 104 to displaythe visible light image and/or the infrared image generated by thermalimaging camera 10. In some additional examples, as described in greaterdetail below, processor 106 may also control range imaging camera module102 to determine distances between the thermal imaging camera and aplurality of different points (e.g., objects) in a target scene.Processor 106 may use these distances, which may be referred to asdistance-to-target data, to perform various processing functions. Forexample, processor 106 may use the distances-to-target data to helpresolve a parallax between the thermal image and the visible light imagegenerated by thermal imaging camera 10 by shifting different portions ofthe thermal image and/or the visible light image based on the distanceto target data. As another example, processor 106 may use thedistance-to-target data to help focus the infrared optics of thermalimaging camera 10.

Infrared camera module 100 may be configured to receive infrared energyemitted by a target scene and to focus the infrared energy on aninfrared detector for generation of infrared energy data, e.g., that canbe displayed in the form of an infrared image on display 104 and/orstored in memory 110. Infrared camera module 100 can include anysuitable components for performing the functions attributed to themodule herein. In the example of FIG. 3, infrared camera module isillustrated as including infrared lens assembly 14 and infrared detector114. As described above with respect to FIGS. 1 and 2, infrared lensassembly 14 includes at least one lens that takes infrared energyemitted by a target scene and focuses the infrared energy on infrareddetector 114. Infrared detector 114 responds to the focused infraredenergy by generating an electrical signal that can be converted anddisplayed as an infrared image on display 104.

Infrared detector 114 may include one or more focal plane arrays (FPA)that generate electrical signals in response to infrared energy receivedthrough infrared lens assembly 14. Each FPA can include a plurality ofinfrared sensor elements including, e.g., bolometers, photon detectors,or other suitable infrared sensor elements. In operation, each sensorelement, which may each be referred to as a sensor pixel, may change anelectrical characteristic (e.g., voltage or resistance) in response toabsorbing infrared energy received from a target scene. In turn, thechange in electrical characteristic can provide an electrical signalthat can be received by processor 106 and processed into an infraredimage displayed on display 104.

For instance, in examples in which infrared detector 114 includes aplurality of bolometers, each bolometer may absorb infrared energyfocused through infrared lens assembly 14 and increase in temperature inresponse to the absorbed energy. The electrical resistance of eachbolometer may change as the temperature of the bolometer changes.Processor 106 may measure the change in resistance of each bolometer byapplying a current (or voltage) to each bolometer and measure theresulting voltage (or current) across the bolometer. Based on thesedata, processor 106 can determine the amount of infrared energy emittedby different portions of a target scene and control display 104 todisplay a thermal image of the target scene.

Independent of the specific type of infrared sensor elements included inthe FPA of infrared detector 114, the FPA array can define any suitablesize and shape. In some examples, infrared detector 114 includes aplurality of infrared sensor elements arranged in a grid pattern suchas, e.g., an array of sensor elements arranged in vertical columns andhorizontal rows. In various examples, infrared detector 114 may includean array of vertical columns by horizontal rows of, e.g., 16×16, 50×50,160×120, 120×160 or 640×480. In other examples, infrared detector 114may include a smaller number of vertical columns and horizontal rows(e.g., 1×1), a larger number vertical columns and horizontal rows (e.g.,1000×1000), or a different ratio of columns to rows.

During operation of thermal imaging camera 10, processor 106 can controlinfrared camera module 100 to generate infrared image data for creatingan infrared image. Processor 106 can generate a “frame” of infraredimage data by measuring an electrical signal from each infrared sensorelement included in the FPA of infrared detector 114. The magnitude ofthe electrical signal (e.g., voltage, current) from each infrared sensorelement may correspond to the amount of infrared radiation received byeach infrared sensor element, where sensor elements receiving differentamounts of infrared radiation exhibit electrical signal with differentmagnitudes. By generating a frame of infrared image data, processor 106captures an infrared image of a target scene at a given point in time.

Processor 106 can capture a single infrared image or “snap shot” of atarget scene by measuring the electrical signal of each infrared sensorelement included in the FPA of infrared detector 114 a single time.Alternatively, processor 106 can capture a plurality of infrared imagesof a target scene by repeatedly measuring the electrical signal of eachinfrared sensor element included in the FPA of infrared detector 114. Inexamples in which processor 106 repeatedly measures the electricalsignal of each infrared sensor element included in the FPA of infrareddetector 114, processor 106 may generate a dynamic thermal image (e.g.,a video representation) of a target scene. For example, processor 106may measure the electrical signal of each infrared sensor elementincluded in the FPA at a rate sufficient to generate a videorepresentation of thermal image data such as, e.g., 30 Hz or 60 Hz.Processor 106 may perform other operations in capturing an infraredimage such as sequentially actuating a shutter (not illustrated) to openand close an aperture of infrared lens assembly 14, or the like.

Each infrared sensor element included in the FPA of infrared detector114 may correspond to a different portion of a target scene beingcaptured. For example, during operation, infrared energy from athree-dimensional target scene may be received by infrared lens assembly14 and focused onto a two-dimensional infrared detector 114 so thatinfrared energy from different portions of the target scene are receivedby different sensor elements of infrared detector 114. In such anexample, each infrared sensor element of infrared detector 114 mayreceive infrared energy from a different portion of the target scenebeing captured. The electrical signal associated with a particularinfrared sensor element may correspond to where within the target scenethe infrared energy was emitted from and can thus be used to calculatethe temperature profile at the corresponding point within the capturedtarget scene. In such an example, the target scene may be divided into anumber of portions corresponding to the number of infrared sensorelements in infrared detector 114, e.g., so that there is a one-to-onecorrespondence between portions of a target scene and infrared detectorelements. Processor 106 can measure the magnitude of the electricalsignal (e.g., voltage, current) from each infrared sensor element anddetermine the amount of infrared energy from the portion of the targetscene associated with each respective sensor element.

With each sensor element of infrared detector 114 functioning as asensor pixel, processor 106 can generate a two-dimensional image orpicture representation of the infrared radiation from a target scene bytranslating changes in an electrical characteristic (e.g., resistance)of each sensor element into a time-multiplexed electrical signal thatcan be processed, e.g., for visualization on display 104 and/or storagein memory 110. Processor 106 may perform computations to convert rawinfrared image data into scene temperatures including, in some examples,colors corresponding to the scene temperatures.

Processor 106 may control display 104 to display at least a portion ofan infrared image of a captured target scene. In some examples,processor 106 controls display 104 so that the electrical response ofeach sensor element of infrared detector 114 is associated with a singlepixel on display 104. In other examples, processor 106 may increase ordecrease the resolution of an infrared image so that there are more orfewer pixels displayed on display 104 than there are sensor elements ininfrared detector 114. Processor 106 may control display 104 to displayan entire infrared image (e.g., all portions of a target scene capturedby thermal imaging camera 10) or less than an entire infrared image(e.g., a lesser port of the entire target scene captured by thermalimaging camera 10). Processor 106 may perform other image processingfunctions, as described in greater detail below.

Although not illustrated on FIG. 3, thermal imaging camera 10 mayinclude various signal processing or conditioning circuitry to convertoutput signals from infrared detector 114 into a thermal image ondisplay 104. Example circuitry may include a bias generator formeasuring a bias voltage across each sensor element of infrared detector114, analog-to-digital converters, signal amplifiers, or the like.Independent of the specific circuitry, thermal imaging camera 10 may beconfigured to manipulate data representative of a target scene so as toprovide an output that can be displayed, stored, transmitted, orotherwise utilized by a user.

As briefly noted above, thermal imaging camera 10 includes range imagingcamera module 102. Range imaging camera module 102 may be configured toreceive visible light energy from a target scene and to focus thevisible light energy on a visible light detector for generation ofvisible light energy data, e.g., that can be displayed in the form of avisible light image on display 104 and/or stored in memory 110. Rangeimaging camera module 102 may also be configured to determine distancesbetween thermal imaging camera 10 and a plurality of different portionsof a target scene being captured. For example, range imaging cameramodule 102 may determine the distance between thermal imaging camera 10and different objects in the target scene, where the different object inthe target scene are arranged at different distances relative to thermalimaging camera 10.

Range imaging camera module 102 can include any suitable components forperforming the functions attributed to the module herein. In the exampleof FIG. 3, range imaging camera module 102 is illustrated as includingvisible light lens assembly 16 and a range imaging system 116. Rangeimaging system 116 may be implemented so as to depth image athree-dimensional target scene about an optical axis of visible lightlens assembly 16.

In one example, range imaging system 116 includes a structured lightcamera assembly that is configured to project a light pattern onto atarget scene and determine from the projected light distances betweenthermal imaging camera 10 and different points (e.g., objects) in thetarget scene. In such an example, range imaging system 116 may beconfigured to project a light pattern (e.g., light emitted through ashaped surface) onto a target scene, receive light reflected offdifferent surfaces in the target scene, and determine from thedisplacement of different portions of the patterned light (e.g.,displacement of different portions of one or more stripes) distancesbetween thermal imaging camera 10 and different points (e.g., objects)in the target scene.

In another example, range imaging system 116 includes a light fieldcamera assembly that is configured receive visible light from a targetscene and determine from the received light distances between thermalimaging camera 10 and different points (e.g., objects) in the targetscene. For example, the light field camera may be configured to receivevisible light from a target scene, separate the received light into,e.g., color, intensity, and direction data, and determine from theseparated data distances between thermal imaging camera 10 and differentpoints (e.g., objects) in the target scene.

In another example, range imaging system 116 may include two or morevisible light camera modules (e.g., each having a visible light detectorand visible light optics) that are offset from one another on housing12. In operation, each visible light camera module may receive visiblelight from a target scene such that there is a parallax between avisible light image associated with (e.g., captured by) each visiblelight camera module. The parallax may be proportional to the distancebetween thermal imaging camera 10 and different points (e.g., objects)in the target scene. According, in such an example, thermal imagingcamera 10 may determine the distance between thermal imaging camera 10and different points (e.g., objects) in the target scene based on thevisible light received by each of the two or more visible light cameramodules and the parallax associated with the offset between the two ormore visible light camera modules.

Range imaging system 116 may include any other suitable system(s) fordetermining distance-to-target data between thermal imaging camera 10and different objects in a target scene. In yet another example, rangeimaging system 116 includes a time-of-flight (TOF) system. An exampleTOF system is a ZCam™ three-dimensional camera manufactured by 3DVSystems, although types of TOF systems may be used in accordance withthe disclosure.

In general, a TOF system operates by emitting optical energy toward atarget scene and then analyzing a signal reflected back by differentobjects in the target scene. In some examples, the TOF system emitspulses of optical energy and determines the amount of time it takes forthe emitted pulses to be detected as optical energy that reflects atleast partially off of objects in the target scene. In these examples,the distance to different objects in the target scene can be determinedbased on the velocity of light and the measured time-of-flight. In otherexamples, the TOF system emits optical energy having a known phase anddetermines the distance to different objects in the target scene byexamining the phase-shift in the signal reflected at least partially offof object in the target scene.

An example TOF system is illustrated in FIG. 3, where range imagingsystem 116 includes a visible light detector 118, and an optical energysource 120. Optical energy source 120 is configured to emit opticalenergy toward a target scene, while visible light detector 118 isconfigured receive optical energy from the target scene. Visible lightdetector 118 may include a plurality of sensor elements such as, e.g.,CMOS detectors, CCD detectors, PIN diodes, avalanche photo diodes, orthe like, and the sensor elements may be referred to as visible lightsensor elements. Optical energy source 120 may include one or more lightemitting diodes, laser diodes, or the like. Optical energy source 120may emit light at any suitable wavelengths including, e.g., in thevisible light spectrum, infrared spectrum, and/or ultraviolet spectrum.Therefore, although visible light detector 118 is described below asincluding visible light sensor elements, it should be appreciated thatthe visible light sensor elements may be configured to detect energy inwavelengths other than the visible light spectrum and the disclosure isnot limited in this respect.

In operation, optical energy source 120 can be periodically energizedunder the control of processor 106. Optical energy emitted by opticalenergy source 120 may reflect off the surfaces of different objects inthe target scene. This reflected optical energy can pass through a lensassembly (e.g., visible light lens assembly 16) and focus on visiblelight detector 118. When the reflected optical energy impinges upon thevisible light sensor elements of visible light detector 118, photonswithin the photodetectors may be released and converted into a detectioncurrent. Processor 106 can process this detection current to form adepth image (e.g., depth map) of the target scene.

For example, reflected energy received by each visible light sensorelement may include signal amplitude information indicative of theluminosity of a specific portion of the target scene and phase shiftinformation indicative of the distance between the visible light sensorelement and object(s) in the portion of the target scene captured by thevisible light sensor element. Processor 106 can compare the shift inphase between the emitted optical energy and the detected optical energyand determine therefrom the distance between each visible light sensorelement and object(s) in the portion of the target scene captured byeach visible light sensor element. Processor 106 may convert thesedistances directly into physical units such as inches, feet, yards, orthe like, e.g., for visualization on display 104 and/or storage inmemory 110. The determined distances between each visible light sensorelement of visible light detector 118 and object(s) in the portion ofthe target scene captured by each respective visible light sensorelement may be referred to as distance-to-target data.

During operation of thermal imaging camera 10, processor 106 can controlrange imaging camera module 102 to generate visible light data from acaptured target scene for creating a visible light image. The visiblelight data may include luminosity data indicative of the color(s)associated with different portions of the captured target scene and/orthe magnitude of light associated with different portions of thecaptured target scene. The visible light data may also includedistance-to-target data indicative of the distance between thermalimaging camera 10 and objects in different portions of the capturedtarget scene. Processor 106 can generate a “frame” of visible lightimage data by measuring the response of each visible light sensorelement of thermal imaging camera 10 a single time. By generating aframe of visible light data, processor 106 captures visible light imageof a target scene at a given point in time. Processor 106 may alsorepeatedly measure the response of each visible light sensor element ofthermal imaging camera 10 so as to generate a dynamic thermal image(e.g., a video representation) of a target scene, as described abovewith respect to infrared camera module 100.

As with the infrared sensor elements of infrared detector 114, eachvisible light sensor element of range imaging camera module 102 maycorrespond to a different portion of a target scene being captured. Forexample, each visible light sensor element of range imaging cameramodule 102 may receive energy from a different portion of the targetscene being captured. The electrical signal associated with each visiblelight sensor element may then correspond to where within the targetscene the light energy originated from (e.g., where an optical pulsereflected from) and can thus be used to create a visible light image atthe corresponding point within the captured target scene. In such anexample, the target scene may be divided into a number of portionscorresponding to the number of visible light sensor elements in rangeimaging camera module 102, e.g., so that there is a one-to-onecorrespondence between portions of a target scene and visible lightsensor elements. The number of visible light sensor elements may be thesame as or different than the number of infrared light sensor elements.

With each sensor element of range imaging camera module 102 functioningas a sensor pixel, processor 106 can generate a two-dimensional image orpicture representation of the visible light from a target scene bytranslating an electrical response of each sensor element into atime-multiplexed electrical signal that can be processed, e.g., forvisualization on display 104 and/or storage in memory 110.

Processor 106 may control display 104 to display at least a portion of avisible light image of a captured target scene. In some examples,processor 106 controls display 104 so that the electrical response ofeach sensor element of range imaging camera module 102 is associatedwith a single pixel on display 104. In other examples, processor 106 mayincrease or decrease the resolution of a visible light image so thatthere are more or fewer pixels displayed on display 104 than there aresensor elements in range imaging camera module 102. Processor 106 maycontrol display 104 to display an entire visible light image (e.g., allportions of a target scene captured by thermal imaging camera 10) orless than an entire visible light image (e.g., a lesser port of theentire target scene captured by thermal imaging camera 10).

Processor 106 may use distance-to-target data determined via rangeimaging camera module 102 to perform a variety of different functions.In one example, processor 106 may control display 104 to display a depthimage of a captured target scene. The depth image may graphically and/ortextually indicate distances between thermal imaging camera 10 anddifferent portions of the target scene. For example, display 104 maydisplay an infrared and/or visible light image overlaid with numbersindicating distances between thermal imaging camera 10 and differentportions of the target scene captured by the image. In another example,display 104 may display a shaded infrared and/or visible light image,where different shading indicates different distances between thermalimaging camera 10 and the respective shaded portion of the target scenecaptured by the image. In still other examples, processor 106 maycontrol display 104 to indicate distance-to-target data for a specificportion of a target scene captured by the image in response to userselection of the specific portion of the target scene. In this example,a user may select a portion of a visible light image and/or an infraredimaged displayed on display 104, and processor 106 may provide anindication of the distance(s) between thermal imaging camera 10 and theselected portion of the image in response to the user selection.

As noted above, processor 106 may control display 104 to concurrentlydisplay at least a portion of the visible light image captured bythermal imaging camera 10 and at least a portion of the infrared imagecaptured by thermal imaging camera 10. Such a concurrent display may beuseful in that an operator may reference the features displayed in thevisible light image to help understand the features concurrentlydisplayed in the infrared image, as the operator may more easilyrecognize and distinguish different real-world features in the visiblelight image than the infrared image. In various examples, processor 106may control display 104 to display the visible light image and theinfrared image in side-by-side arrangement, in a picture-in-picturearrangement, where one of the images surrounds the other of the images,or any other suitable arrangement where the visible light and theinfrared image are concurrently displayed.

For example, processor 106 may control display 104 to display thevisible light image and the infrared image in a fused arrangement. In afused arrangement, the visible light image and the infrared image may besuperimposed on top of one another. An operator may interact with userinterface 108 to control the transparency or opaqueness of one or bothof the images displayed on display 104. For example, the operator mayinteract with user interface 108 to adjust the infrared image betweenbeing completely transparent and completely opaque and also adjust thevisible light image between being completely transparent and completelyopaque. Such an example fused arrangement, which may be referred to asan alpha-blended arrangement, may allow an operator to adjust display104 to display an infrared-only image, a visible light-only image, ofany overlapping combination of the two images between the extremes of aninfrared-only image and a visible light-only image.

When processor 106 controls display 104 to concurrently display thevisible light image and the infrared image, processor 106 may align theimages to help reduce a parallax between the two images. Processor 106may align the images based on distance-to-target data received via rangeimaging camera module 102. For example, processor 106 may aligndifferent portions of the visible light image and/or infrared imagebased on different distance-to-target data associated with the differentportions of the image(s). Each portion of the visible light image maycorrespond with a specific visible light sensor element of the pluralityof visible light sensor elements of range imaging camera module 102,while each portion of the infrared image may correspond with a specificinfrared sensor element of the plurality of infrared sensor elements ofinfrared detector 114. In addition, each portion of the visible lightimage may include corresponding distance-to-target data indicative ofthe distance between thermal imaging camera 10 and the object(s) in theportion of the target scene captured by the specific portion of thevisible light image.

Processor 106 may align different portions of the visible light imageand/or infrared image by shifting portions of one or both of the imagesrelative to the other image. For example, each portion of the visiblelight image and each portion of the infrared image may include positioncoordinates (e.g., Cartesian coordinates) associated with the portion ofthe image. Processor 106 may shift a portion of an image by, e.g.,adding a certain value to or subtracting a certain value from theposition coordinates associated with the portion of image so as todefine new position coordinates associated with the portion of theimage. Processor 106 can control display 104 to display the visiblelight image and the infrared image according to the adjusted positioncoordinates.

In some examples, processor 106 can align the different portions of thevisible light image and/or infrared image based on thedistance-to-target data received via range imaging camera module 102 anddata stored in memory 110. The data may be stored, e.g., in a look-uptable stored in memory 110 that associates different distance-to-targetvalues with different parallax correction values. In another example,the data may be stored in the form of an equation that associatesdifferent distance-to-target values with different parallax correctionvalues. Using the distance-to-target value associated with a specificportion of an image, processor may determine the parallax correctionvalue associated with the portion of the image upon reference to memory110. Processor 106 may then shift the portion of the image based on thedetermined parallax correction and control display 104 so as to displaythe shifted portion of the image in alignment with the other image.

Depending on the configuration of thermal imaging camera 10, a featurein a target scene that is relatively close to the thermal imaging cameramay exhibit more parallax between the visible light image and theinfrared image than a feature in the target scene that is farther awayfrom the thermal imaging camera. In this example, processor 106 mayshift portions of the visible light image and/or the infrared imagecapturing the relative close feature in the target scene more than theportions of the visible light image and/or infrared image capturing thefarther away feature.

Processor 106 can align a plurality of different portions of a visiblelight image and/or an infrared image so that similar features in thetarget scene captured by the visible light image and/or the infraredimage are displayed in alignment with corresponding portions of theother image. In some examples, processor 106 aligns the visible lightimage and/or infrared image on a pixel-by-pixel basis. For example,processor 106 may shift the alignment of each of the different portionsof the visible light image and/or infrared image displayed on display104, where each portion of the visible light image corresponds with aspecific visible light sensor element of the plurality of visible lightsensor elements of range imaging camera module 102 and each portion ofthe infrared image corresponds with a specific infrared sensor elementof the plurality of infrared sensor elements of infrared detector 114.In other examples, processor 106 may align a plurality of differentportions of the visible light image and/or the infrared image byshifting a larger portion of the visible light image and/or the infraredimage than a portion associated with a specific sensor element. Forexample, processor may determine an average distance-to-target value(e.g., mean, median) corresponding to a portion of an image associatedwith multiple different sensor elements and align the portion of theimage based on a parallax correction value associated with the averagedistance-to-target value. Other techniques for aligning an infraredimage with a visible light image so as to reduce a parallax between theimages are possible and it should be appreciated that the disclosure isnot limited in this respect.

FIGS. 4A and 4B schematically illustrate the processor's 106 alignmentof different portions of visible light image and/or an infrared image.Rectangle 150 corresponds to image 1 and rectangle 160 corresponds toimage 2. In one embodiment, image 1 corresponds to the visible lightimage and image 2 corresponds to the infrared image. In anotherembodiment, image 1 corresponds to the infrared image and image 2corresponds to the visible light images. Although rectangles 150 and 160are shown with different sizes, they also be of the same size. Withspecific reference to FIG. 4A, rectangle 160 is shown to contain threedifferent image portions, portions P1, P2, and P3. Of course, threeportions is only exemplary. Two or more image portions are preferred.The image portions P1, P2, and P3 may correspond to individual sensorelements of the visible light sensor or the FPA. The image portions P1,P2, and P3 may correspond to the larger portions described above, suchas an average distance-to-target value corresponding to a portion of animage associated with multiple sensor elements. In addition, althoughP1, P2, and P3 are shown as being equally sized, they need not be. Withspecific reference to FIG. 4B, it is almost identical to FIG. 4A.However, image portions P1, P2, and P3 of rectangle 160 are shiftedupward (relative to rectangle 150, which remains unmoved) from theircorresponding positions in FIG. 4A. In this example, processor 106 hasshifted image portion P3 further upward than image portion P1, and imageportion P2 remains unmoved. This example demonstrates a situation whereimage portion P3 likely corresponds to a feature in the targeted scenethat is relatively close to the camera since the processor shifted P3upward the largest amount. P1, by comparison, likely corresponds to afeature in the targeted scene that is further away from the camera thanthe feature shown in image portion P3, since the processor 106 shiftedimage portion P1 upward, but by an amount less than that of imageportion P3. Finally, image portion P2 likely corresponds to a feature inthe targeted scene distant from the camera, since the camera did notneed to correct any parallax error associated with image portion P2.

In addition to or in lieu of aligning different portions of a visiblelight image and an infrared image so as to resolve a parallax betweenthe images, processor 106 may perform other functions usingdistance-to-target data determined via range imaging camera module 102.As one example, processor 106 may use distance-to-target data toautomatically adjust a focus of infrared lens assembly 14 and/or visiblelight lens assembly 16. Memory 110 may store data that associatesdifferent distance-to-target values with different focus positions forinfrared lens assembly 14 and/or visible light lens assembly 16.Processor 106 can determine a distance-to-target value for focusinginfrared lens assembly 14 and/or visible light lens assembly 16. Thedistance-to-target value may be an average (e.g., mean, median) of thedifferent distance-to-target values associated with all the differentportions of the target scene, an average (e.g., mean, median) ofdifferent distance-to-target values associated with a specific portionof the target scene (e.g., a center of the scene towards which anoperator is pointing thermal imaging camera 10), or any other suitabledistance-to-target value. Processor 106 can reference memory 110 todetermine a focus position associated with a specific distance-to-targetvalue and control infrared lens assembly 14 and/or visible light lensassembly 16 to set the focus of the lens assembly to the determinedposition. In some examples, processor 106 may control a solenoid, DCmotor, or other electromechanical device to set the focus of the lensassembly to the determined position.

In some examples, a user may select a portion of a visible light imageand/or an infrared image displayed on display 104, and processor 106 mayautomatically adjust a focus of infrared lens assembly 14 and/or visiblelight lens assembly 16 based on the selected feature. For example,display 104 may include a fixed or moveable cursor or other user inputmechanism that allows a user to select a feature of interest in a targetscene displayed on display 104. In response to the selection, processor106 can determine a distance between thermal imaging camera 10 and theselected feature(s) of interest using distance-to-target data determinedvia range imaging camera module 102. Processor 106 may then referencememory 110 to determine a focus position associated with a specificdistance-to-target value and control infrared lens assembly 14 and/orvisible light lens assembly 16 to set the focus of the lens assembly tothe determined position.

Components described as processors within thermal imaging camera 10,including processor 106, may be implemented as one or more processors,such as one or more microprocessors, digital signal processors (DSPs),application specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), programmable logic circuitry, or the like, eitheralone or in any suitable combination.

In general, memory 110 stores program instructions and related datathat, when executed by processor 106, cause thermal imaging camera 10and processor 106 to perform the functions attributed to them in thisdisclosure. Memory 110 may include any fixed or removable magnetic,optical, or electrical media, such as RAM, ROM, CD-ROM, hard or floppymagnetic disks, EEPROM, or the like. Memory 110 may also include aremovable memory portion that may be used to provide memory updates orincreases in memory capacities. A removable memory may also allow imagedata to be easily transferred to another computing device, or to beremoved before thermal imaging camera 10 is used in another application.

An operator may interact with thermal imaging camera 10 via userinterface 108, which may include buttons, keys, or another mechanism forreceiving input from a user. The operator may receive output fromthermal imaging camera 10 via display 104. Display 104 may be configuredto display an infrared-image and/or a visible light image in anyacceptable palette, or color scheme, and the palette may vary, e.g., inresponse to user control. In some examples, display 104 is configured todisplay an infrared image in a monochromatic palette such as grayscaleor amber. In other examples, display 104 is configured to display aninfrared image in a color palette such as, e.g., ironbow, blue-red, orother high contrast color scheme. Combination of grayscale and colorpalette displays are also contemplated.

Different thermal imaging cameras and thermal imaging camera techniqueshave been described in relation to FIGS. 1-3. FIG. 4 is a flow chartillustrating an example method for aligning a visible light image withan infrared image using an apparatus configured to generatedistance-to-target data for a plurality of different portions of ascene. For ease of description, the method of FIG. 5 is described asexecuted by thermal imaging camera 10 (FIGS. 1-3). In other examples,however, the method of FIG. 5 may be executed by apparatuses withdifferent configurations, as described herein.

As shown in FIG. 5, processor 106 (FIG. 3) receives visible light imagedata representative of a visible light image of a scene along visiblelight optical axis 26 in response to depressing trigger control 20(FIG. 1) (200). During operation, visible light energy from the sceneenters visible light lens assembly 16, which focuses the visible lightenergy on visible light detector 118. Visible light detector 118 mayinclude a plurality of visible light sensor element such as, e.g., CMOSdetectors, which may each generate a detection signal in response toreceiving the visible light energy focused through visible light lensassembly 16. In such an example, each visible light sensor element ofthe plurality of visible light sensor elements may receive visible lightenergy from a different portion of the scene so that the detectionsignal associated with each visible light sensor element may correspondto where within the scene the visible light energy originated. Processor106 may receive the detection signals from multiple visible light sensorelements (e.g., all of the visible light sensor elements) so as toreceive visible light image data associated with a plurality ofdifferent portions of the scene captured by the visible light image.

In some examples, processor 106 also receives data indicative of thedistance between thermal imaging camera 10 and different features ineach of the plurality of different portions of the scene captured by thevisible light image. In one example, processor 106 controls opticalenergy source 120 (FIG. 3) so as to emit optical energy toward thedifferent features in the scene. Optical energy reflected back from thedifferent features in the scene may be received by each visible lightsensor element of the plurality of visible light sensor elements ofvisible light detector 118. Depending on the configuration of thermalimaging camera 10, the reflected optical energy received by the visiblelight sensor elements may include phase shift information indicative ofthe distance between each visible light sensor element and feature(s) inthe portion of the scene associated with each visible light sensorelement (e.g., features in the portion of the scene corresponding towhere within the scene the optical energy reflected from). Processor 106may compare the shift in phase between the emitted optical energy andthe detected optical energy and determine therefrom the distance betweeneach visible light sensor element and feature(s) in the portion of thescene captured by each visible light sensor element. In this manner,processor 106 may receive distance-to-target data associated with aplurality of different portions of a scene captured by a visible lightimage.

With further reference to FIG. 5, processor 106 also receives infraredimage data representative of an infrared image of a scene along infraredoptical axis 22 in response to depressing trigger control 20 (FIG. 1)(202). In some examples, processor 106 receives infrared image data(202) and visible light image data (200) from an infrared image and avisible light image that are captured at substantially the same time,although the infrared image and visible light image may be captured atdifferent times as well. In either example, infrared energy from a sceneenters infrared lens assembly 14, which focuses the infrared energy oninfrared detector 114. Infrared detector 114 may include a plurality ofinfrared sensor elements such as, e.g., bolometers, which may change anelectrical characteristic (e.g., resistance) in response to receivingthe infrared energy focused through infrared lens assembly 14. In suchan example, each infrared sensor element of the plurality of infraredsensor elements may receive infrared energy from a different portion ofthe scene so that an electrical signal associated with each infraredsensor element may correspond to where within the scene the infraredenergy originated. Processor 106 may receive electrical signals frommultiple infrared sensor elements (e.g., all of the infrared sensorelements) so as to receive infrared data associated with a plurality ofdifferent portions of the scene captured by the infrared image.

In the example technique of FIG. 5, processor 106 aligns each of aplurality of different portions of a visible light image with aninfrared image based on distance-to-target data so as to correct aparallax error between the visible light image and the infrared image(204). For example, processor 106 may align specific portions of thevisible light image with specific portions of the infrared image basedon different distance-to-target data associated with each of thedifferent portions of the visible light image. Each portion of thevisible light image may correspond with a specific visible light sensorelement of the plurality of visible light sensor elements of visiblelight detector 118, while each portion of the infrared image maycorrespond with a specific infrared sensor element of the plurality ofinfrared sensor elements of infrared detector 114.

In some examples, processor 106 aligns specific portions of the visiblelight image with specific portions of the infrared image based on thedistance-to-target data received via range imaging camera module 102 anddata stored in memory 110. The data may be stored, e.g., in the form ofa look-up table, an equation, or other form that associates differentdistance-to-target values with different parallax correction values.Using the distance-to-target value associated with a specific portion ofthe visible light image, processor 106 may determine the parallaxcorrection value associated with a specific portion of the visible lightimage upon reference to memory 110. Processor 106 may then shift theportion of the visible light image based on the determined parallaxcorrection value so that the shifted portion of the visible light imagealigns with a corresponding portion of the infrared image such that ascene feature captured by both images can be displayed in registration(e.g., without parallax) upon concurrent display of both images.

According to the example technique of FIG. 5, processor 106 alsocontrols display 104 to concurrently display at least a portion of aninfrared image and at least a portion of a visible light image such thatcorresponding portions of the two images that capture the samefeature(s) in the scene are displayed in alignment (206). In variousexamples, processor 106 may control display 104 to display the visiblelight image and the infrared image in side-by-side arrangement, in apicture-in-picture arrangement, or in another arrangement such thatcorresponding portions of the two images that capture the samefeature(s) in the scene are displayed in alignment. Correspondingportions of the infrared image and the visible light image may bedisplayed in alignment such that there is substantially no parallaxerror between the portions of the two images. For example, if theportions of the infrared image and visible light image are displayed inoverlapping alignment (e.g., one on top of the other), the features fromthe scene that are captured by both images may substantially (e.g.,exactly) overlap with one another. If the portions of the infrared imageand visible light image are displayed in adjacent alignment (e.g.,side-by-side alignment, picture-in-picture alignment) the features fromthe scene that are captured by both images may substantially (e.g.,exactly) align with one another such that there is no shift or offsetbetween the features between the portion of the displayed visible lightimage and the portion of the displayed infrared image.

While processor 106 can controls display 104 to concurrently display atleast a portion of an infrared image and at least a portion of a visiblelight image (206) in any suitable arrangement, a picture-in-picturearrangement may help an operator to easily focus and/or interpret athermal image by displaying a corresponding visible image of the samescene in adjacent alignment. FIG. 6 is a conceptual illustration of oneexample picture-in-picture type display of a visual image 240 and aninfrared image 242. In the example of FIG. 6, visual image 240 surroundsinfrared image 242, although in other examples infrared image 242 maysurround visual image 240, or visual image 240 and infrared image 242may have different relative sizes or shapes than illustrated and itshould be appreciated that the disclosure is not limited in thisrespect.

Example thermal image cameras and related techniques have beendescribed. The techniques described in this disclosure may also beembodied or encoded in a computer-readable medium, such as anon-transitory computer-readable storage medium, containinginstructions. Instructions embedded or encoded in a computer-readablestorage medium may cause a programmable processor, or other processor,to perform the method, e.g., when the instructions are executed.Computer readable storage media may include random access memory (RAM),read only memory (ROM), programmable read only memory (PROM), erasableprogrammable read only memory (EPROM), electronically erasableprogrammable read only memory (EEPROM), flash memory, a hard disk, aCD-ROM, a floppy disk, a cassette, magnetic media, optical media, orother computer readable media.

Various examples have been described. These and other examples arewithin the scope of the following claims.

The invention claimed is:
 1. A camera comprising: a range imaging cameramodule configured to capture a visible-light (VL) image of a scene alonga first optical axis and generate distance-to-target data for the scene,the VL image including a plurality of different portions that eachcorrespond to a different portion of the scene, and thedistance-to-target data comprising data representative of distances fromthe ranging imaging camera module to different objects located atdifferent depths in the scene, the distance-to-target data including adistance-to-target value for each of the different portions of thescene; an infrared camera module configured to capture an infrared (IR)image of the scene along a second optical axis, the second optical axisbeing offset from the first optical axis so that the IR image of thescene is from a different point of view than the VL image therebycausing a parallax error; a display; and a processor configured to aligneach of the plurality of different portions of the VL image with the IRimage by shifting different portions of the VL image different amountsusing the distance-to-target value associated with each portion of theVL image being shifted so as to correct the parallax error between theVL image and the IR image, and control the display to concurrentlydisplay at least a portion of the VL image in alignment with at least aportion of the IR image.
 2. The camera of claim 1, wherein the rangeimaging camera module comprises a time-of-flight camera system.
 3. Thecamera of claim 2, wherein the time-of-flight camera system comprises aVL lens defining the first optical axis, a VL detector associated withthe VL lens, and an optical energy source configured to emit opticalenergy for determining the distance-to-target data.
 4. The camera ofclaim 1, wherein the range imaging camera module comprises a VL detectorthat includes a plurality of VL sensor elements, and each of theplurality of different portions of the VL image correspond to adifferent one of the plurality of VL sensor elements.
 5. The camera ofclaim 4, wherein the distance-to-target value for each portion of the VLimage corresponds to a specific VL sensor element, and thedistance-to-target value comprises a value representative of thedistance between the range imaging camera module and a point in thescene captured by the specific VL sensor element associated with thedistance-to-target value.
 6. The camera of claim 4, wherein theprocessor is configured to align each of the plurality of differentportions of the VL image by at least determining a parallax correctionvalue associated with each portion of the VL image corresponding to aspecific VL sensor element.
 7. The camera of claim 6, wherein theprocessor is configured to determine the parallax correction value basedupon at least one of an equation or a look-up table associatingdifferent distance-to-target values with different parallax correctionvalues.
 8. The camera of claim 1, wherein the processor is configured toalign each of the plurality of different portions of the VL image byadjusting position coordinates associated with each of the plurality ofdifferent portions of the VL image.
 9. The camera of claim 1, whereinthe processor is configured to control the display to concurrentlydisplay at least one of the VL image and the IR image surrounded by theother of the images so as to effect a picture-in-picture display of thescene.
 10. A method comprising: receiving visible-light (VL) image datarepresentative of a VL image of a scene captured via a range imagingcamera module along a first optical axis and distance-to-target datagenerated by the range imaging camera module for the scene, the VL imageincluding a plurality of different portions that each correspond to adifferent portion of the scene, and the distance-to-target datacomprising data representative of distances from the ranging imagingcamera module to different objects located at different depths in thescene, the distance-to-target data including a distance-to-target valuefor each of the different portions of the scene; receiving infrared (IR)image data representative of an IR image of the scene captured via an IRcamera module along a second optical axis, the second optical axis beingoffset from the first optical axis so that the IR image of the scene isfrom a different point of view than the VL image thereby causing aparallax error; aligning each of the plurality of different portions ofthe VL image with the IR image by shifting different portions of the VLimage different amounts based on the distance-to-target value associatedwith each portion of the VL image being shifted so as to correct theparallax error between the VL image and the IR image; and concurrentlydisplaying at least a portion of the VL image in alignment with at leasta portion of the IR image.
 11. The method of claim 10, wherein the rangeimaging camera module comprises a time-of-flight camera system.
 12. Themethod of claim 10, wherein receiving VL image data comprises receivingVL image data from a plurality of VL sensor elements, each of theplurality of VL sensor elements corresponding to a different one of theplurality of VL image portions.
 13. The method of claim 12, wherein thedistance-to-target value associated with each portion of the VL imagecorresponds to a specific VL sensor element, and the distance-to-targetvalue comprises a value representative of the distance between the rangeimaging camera module and a point in the scene captured by the specificVL sensor element associated with the distance-to-target value.
 14. Themethod of claim 12, wherein aligning each of the plurality of differentportions of the VL image comprises determining a parallax correctionvalue associated with each portion of the VL image corresponding to aspecific VL sensor element.
 15. The method of claim 14, whereindetermining the parallax correction value comprises referencing at leastone of an equation or a look-up table associating differentdistance-to-target values with different parallax correction values. 16.The method of claim 10, wherein the IR camera module comprises an IRlens defining the second optical axis, and an IR detector associatedwith the IR lens.
 17. The method of claim 10, wherein concurrentlydisplaying at least a portion of the VL image in alignment with at leasta portion of the IR image comprises concurrently displaying at least oneof the VL image and the IR image surrounded by the other of the imagesto effect a picture-in-picture display of the scene.
 18. Anon-transitory computer-readable storage medium comprising instructionsthat cause a programmable processor to: receive visible-light (VL) imagedata representative of a VL image of a scene captured via a rangeimaging camera module along a first optical axis and distance-to-targetdata generated by the range imaging camera module for the scene, the VLimage including a plurality of different portions that each correspondto a different portion of the scene, and the distance-to-target datacomprising data representative of distances from the ranging imagingcamera module to different objects located at different depths in thescene, the distance-to-target data including a distance-to-target valuefor each of the different portions of the scene; receive infrared (IR)image data representative of an IR image of the scene captured via an IRcamera module along a second optical axis, the second optical axis beingoffset from the first optical axis so that the IR image of the scene isfrom a different point of view than the VL image thereby causing aparallax error; align each of the plurality of different portions of theVL image with the IR image by shifting different portions of the VLimage different amounts based on the distance-to-target value associatedwith each portion of the VL image being shifted so as to correct theparallax error between the VL image and the IR image; and control adisplay to concurrently display at least a portion of the VL image inalignment with at least a portion of the IR image.
 19. Thenon-transitory computer-readable storage medium of claim 18, wherein theinstructions that cause the programmable processor to receivevisible-light (VL) image data comprise instructions that cause theprogrammable processor to receive VL image data from a plurality of VLsensor elements, each of the plurality of VL sensor elementscorresponding to a different one of the plurality of VL image portions.20. The non-transitory computer-readable storage medium of claim 19,wherein the distance-to-target value associated with each portion of theVL image corresponds to a specific VL sensor element, and thedistance-to-target value comprises a value representative of thedistance between the range imaging camera module and a point in thescene captured by the specific VL sensor element associated with thedistance-to-target value.
 21. The non-transitory computer-readablestorage medium of claim 19, wherein the instructions that cause theprogrammable processor to align each of the plurality of differentportions of the VL image comprise instructions that cause theprogrammable processor to determine a parallax correction valueassociated with each portion of the VL image corresponding to a specificVL sensor element.
 22. The non-transitory computer-readable storagemedium of claim 21, wherein the instructions that cause the programmableprocessor to determine a parallax correction value comprise instructionsthat cause the programmable processor to reference at least one of anequation or a look-up table associating different distance-to-targetvalues with different parallax correction values.
 23. The non-transitorycomputer-readable storage medium of claim 18, wherein the instructionsthat cause the programmable processor to control the display compriseinstructions that cause the programmable processor to control thedisplay to concurrently display at least one of the VL image and the IRimage surrounded by the other of the images to effect apicture-in-picture display of the scene.
 24. The camera of claim 1,wherein the VL image comprises a plurality of pixels havingdistance-to-target values associated with each pixel, and the processoris configured to align each of the plurality of different portions ofthe VL image with the IR image by shifting each pixel of the VL image anamount based on the distance-to-target value associated with that pixelso as to correct the parallax error.